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FREQUENCY OF RESPONSE AND PROBABILITY 
OF ACTION 

A LL psychologists study behavior — even 
those who believe this to be merely a step 
toward a subject matter of another sort. 
All psychologists therefore face certain important 
common problems. The “pure” experimental study 
of behavior in either the field or the laboratory is 
by its very nature concerned with problems of this 
sort. Any progress it may make toward solutions 
should be of interest to everyone who deals with 
behavior for any reason whatsoever. 

As an example, let us consider a concept which, 
in the most general terms, may be called “prob- 
ability of action.” Behavior which has already oc- 
curred and may never be repeated is of limited in- 
terest. Psychologists are usually especially con- 
cerned with the future of the organisms they study. 
They want to predict what an individual will do or 
at least to specify some of the features which his 
behavior will exhibit under certain circumstances. 
They also frequently want to control behavior or to 
impress certain features upon it. But what sort of 
subject matter is juture behavior? How is it rep- 
resented in the organism under observation? 

Generally it is argued or implied that when we 
predict or arrange a future course of action, we are 
dealing with some contemporary state of the or- 
ganism which represents the specified action before 
it has taken place. Thus, we speak of tendencies 
or readiness to behave as if they corresponded to 
something in the organism at the moment. We 
give this “something” many names — from the pre- 
paratory set of experimental psychology to the 
Freudian wish. Habits and instincts, dispositions 
and predispositions, attitudes, opinions, even per- 
sonality itself, are all attempts to represent in the 
present organism something of its future behavior. 

1 Adapted from a lecture at the Thirteenth International 
Congress of Psychology in Stockholm, Sweden, July 1951. 


Probability of action has been given the physical 
status of a thing. It has been, so to speak, em- 
bodied in the organism — in the neurological or 
psychic states or events with which habits, wishes, 
attitudes, and so on may be identified. This solu- 
tion has forced us to assign extraneous properties 
to behavior which are not supported by the data 
and which have been quite misleading. 

The physical referent of a probability must be 
among our data or the problem would not have 
been so persistent. The mistake we make is in 
looking for it as a property of a single event, oc- 
cupying only one point in time. As the mathe- 
maticians have noted, perhaps not unanimously, a 
probability is a way of representing a frequency of 
occurrence. In the program of research to be sum- 
marized and exemplified here, probability of action 
has been attacked experimentally by studying the 
repeated appearance of an act during an appreci- 
able interval of time. 

Frequency of response is emphasized by most of 
the concepts which have foreshadowed an explicit 
recognition of probability as a datum. An organ- 
ism possesses a “habit” to the extent that a certain 
form of behavior is observed with a special fre- 
quency — attributable to events in the history of 
the individual. It possesses an “instinct” to the 
extent that a certain form of behavior is observed 
with a special frequency — in this case because of 
membership in a given species. An “attitude” ex- 
presses a special frequency of a number of forms 
of behavior. These frequencies are the observable 
facts and may be studied as such rather than as 
evidence for the embodiment of probability in 
neural or psychic states. 

Dozens of less technical terms serving the same 
purpose point to an all-abiding practical and theo- 
retical interest in frequency of response as a datum. 
We say that someone is a tennis fan if he frequently 
plays tennis under appropriate circumstances. He 
is “enthusiastic” about skating, if he frequently 
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goes skating. He is “greatly interested” in music 
if he plays, listens to, and talks about music fre- 
quently. The “inveterate” gambler gambles fre- 
quently. The “highly sexed” frequently engage in 
sexual behavior. The linguistic effect of terms of 
this sort — as of the more technical terms — is to 
move from an observation of frequency to an in- 
ferred momentary condition. But this should not 
be allowed to influence the direction of our re- 
search. The basic facts can be discovered only by 
examining behavior during appreciable intervals of 
time. 

In designing a laboratory situation in which fre- 
quency of response may be easily examined certain 
considerations must be observed. We must choose 
a sample of behavior which may be so easily identi- 
fied that repeated instances may be reliably counted. 
If our experiment is to be automatic — and there are 
many advantages in making it so — the chosen re- 
sponse must operate an apparatus. The response 
should not require much time, and it should leave 
the organism ready to respond again. These con- 
ditions are rather arbitrary, and our results must 
be qualified accordingly, but they are easily met. 
Sometimes such a response is found ready-made — as 
in studying so-called instinctive behavior. At other 
times it must be, so to speak, constructed. In the 
case of the rat, for example, it has been found con- 
venient to use such a response as depressing a hori- 
zontal bar. In birds — for example, the pigeon — a 
convenient response is pecking a key through a 
small hole in the wall. Each of these responses is 
easily specified and can be readily repeated. The 
pigeon may peck the key, for example, as rapidly 
as fifteen times per second. 

To record the frequency of such a response we 
could, of course, use the standard polygraph, but 
another sort of curve has proved to be much more 
convenient. A pen is arranged to move one step 
across a strip of paper each time the organism re- 
sponds. The result is a steplike diagonal line. 
Frequency is thus converted into the slope of the 
recorded line. Coordinates are chosen which con- 
vert the commonest frequencies into convenient 
slopes, If the organism is responding rapidly, the 
line is fairly steep. If it responds slowly, the slope 
is low. If it does not respond at all, the pen draws 
a horizontal line. With a little practice it is easy 
to estimate frequencies from the slopes of such 
graphs and to follow changes in frequency with fair 
accuracy. In Fig, 1 some actual records show the 



Fig. 1. Cumulative curves made by pigeons under vari- 
ous schedules of reinforcement showing relative uniform 
performance over a wide range of rates of responding, 

range of frequencies encountered in the pigeon. 
The separate steps of the pen cannot be seen on 
this scale. One record shows a sustained rate of 
18,000 responses per hour — five responses per sec- 
ond. Another record, by way of comparison, shows 
only 300 responses per hour — one response every 
twelve seconds — yielding a very low slope. 

SCHEDULES OF REWARD 

Frequency of response, so recorded, is a useful 
and significant datum in the experimental analysis 
of behavior. It is a sensitive “dependent variable” 
which has been found to be a function of many 
subtle experimental conditions. The degree of sen- 
sitivity is well illustrated by some results of a cur- 
rent research project 2 under the directorship of Dr. 
Charles Ferster and the author. This work falls 
within the traditional field of learning, a subject 
which demonstrates especially well the importance 
of frequency as a datum. Traditional experiments 
in learning have usually been concerned with 
changes in the character of behavior. The organ- 
ism learns how to do something; it acquires new 
behavior. But the conditions which produce this 
kind of learning continue to have an effect when 

2 This research, carried out under Contract NSori-07631 
between Harvard University and the Office of Naval Re- 
search, has been under the direction of the author. Dr, 
Charles B. Ferster has served as principal investigator. 
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the character or topography of behavior no longer 
changes appreciably. The form of the response re- 
mains unaltered, but its frequency is nevertheless 
still affected. 

Consider, for example, what we call a reward or, 
to use a more technical term suggested by Pavlov, 
a reinforcement. A convenient reinforcement is 
giving food to a hungry organism. If we arrange 
our apparatus so that a small amount of food is 
presented immediately after a selected response is 
made, we observe an immediate increase in the 
frequency of that response. By reinforcing slight 
variations in the form of response, we may mold 
behavior almost as a sculptor molds a lump of clay. 
In our experiments with pigeons the response of 
pecking the key on the wall may be conditioned in 
two or three minutes by selecting behavior which 
progressively approaches the form of this response. 
This is the kind of change traditionally studied in 
the field of learning. But when such behavior has 
been acquired, further reinforcements are not with- 
out effect. They no longer change the form of the 
response, but they maintain it as part of the cur- 
rent repertoire of the organism. This is seen in the 
simple fact that, unless a response continues to be 
reinforced, it disappears in what we call extinction. 
But there are many subtle degrees of probability of 
action between an inevitable response and no re- 
sponse at all. 

The experiments to be described here are part of 
an extensive exploration of the effect of reinforce- 
ments which are only intermittently presented. In 
daily life the connection between a response and its 
consequences is usually not invariable, for it may 
depend upon events which are not fixed. We do 
not always win at cards or dice because the con- 
tingencies are so remotely determined that we call 
them “chance.” We do not inevitably find good ice 
when we' go skating — for very obvious reasons. 
Contingencies which require the participation of 
other people are especially likely to be uncertain. 
We do not always get a good meal in a restaurant 
because chefs are not always predictable. We do 
not always get an answer when we telephone a 
friend because the friend is not always at home. 
We do not always get a fountain pen by reaching 
into our pocket because this result depends upon 
our having put it there. 

In analyzing the effect of intermittent reinforce- 
ment we have to distinguish between many different 
ways in which reinforcements may be scheduled. 



Fig. 2. Fixed-interval reinforcement. The interval (?) is 


In some of these a system outside the organism, 
such as a clock, arranges a connection between the 
movement of the key and the presentation of food. 
For example, a response to the key may be rein- 
forced once every five minutes while all other re- 
sponses are allowed to go unreinforced. In other 
schedules the reinforcements may be arranged by 
the organism itself — for example, we may reinforce 
every fiftieth response. These cases appear to be 
similar in the sense that we reinforce intermittently 
in both, but subtle differences in the schedules lead 
to very different results, many of which are of great 
practical significance. 

We might study the subject by specifying all pos- 
sible schedules or programs of reinforcement and 
by determining the typical performance generated 
by each. In one sense this would give us a com- 
plete picture. But a thorough analysis must go 
further. Why does a given schedule yield a given 
performance? We cannot suppose that it is simply 
because organisms are fond of mathematics. We 
need to examine the way in which a particular 
schedule actually affects the organism, The only 
point of contact between the scheduling mechanism 
and the organism occurs at the moment of rein- 
forcement. Dr. Ferster and I have therefore at- 
tempted to account for the performance character- 
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Fig. 3. Extinction after fixed-interval reinforcement. The record has been broken into two 
segments to avoid undue reduction. 


istic of a given schedule in terms of the conditions 
which prevail at the moment of reinforcement. The 
experimental problem is to separate these condi- 
tions so that their contributions may be evaluated. 

We may represent a schedule in which reinforce- 
ments are arranged by a clock by drawing vertical 
lines on our cumulative graph. In Fig. 2 the lines 
are five minutes apart. A response is reinforced as 
soon as the pen reaches the first line, regardless of 
how many responses have been made. Another re- 
sponse is reinforced when the pen reaches the sec- 
ond line, and so on. In other words, we simply re- 
inforce responses at intervals of approximately five 
minutes. Call this “fixed-interval reinforcement.” 
The organism quickly adjusts with a fairly constant 
rate of responding, which produces a straight line 
with our method of recording. The rate — the slope 
of the line — is a function of several things. It 
varies with difficulty of execution : the more difficult 
the response, the lower the slope. It varies with 
degree of food deprivation: the hungrier the organ- 
ism, the higher the slope. And so on. It will be 
seen, moreover, that such a record is not quite 
straight, After each reinforcement the pigeon 
pauses briefly — in this case for 30 or 40 seconds. 
This is due to the fact that under a fixed-interval 
schedule no response is ever reinforced just after 
reinforcement. The organism is able to form a 
discrimination based upon the stimuli generated in 
the act of eating food. So long as this stimulation 
is effective, the rate is low. Thereafter the or- 
ganism responds at essentially a constant rate. It 
would appear that stimuli due to the mere passage 
of time are not significantly different to the organ- 
ism during the remaining part of the interval. The 
organism cannot, so to speak, tell the difference be- 


tween, say, three and four minutes after reinforce- 
ment under these circumstances. At longer fixed 
intervals — of, say, IS minutes — each segment of 
such a record is a smooth, positively accelerated 
curve. 

A pigeon will continue indefinitely to respond 
when reinforcements are spaced as much as 45 min- 
utes apart. Food is then received too slowly to 
maintain body-weight, so that extra feeding is nec- 
essary between experimental periods. The behavior 
after each reinforcement shows a much slower ac- 
celeration from a low to a high rate. In extinction, 
the effect of self-generated stimuli is seen. Figure 
3 is an example, broken into two segments to show 
details more clearly. The pigeon begins as usual 
at a low rate of responding at A. It has never been 
reinforced at the start of the experiment or immedi- 
ately after another reinforcement. A higher rate 
develops smoothly during the first 20 or 30 min- 
utes. This part of the curve is a fair sample of the 
behavior after each reinforcement on a 45-minute 
schedule. Eventually a rate is reached at which 
reinforcements have been most often received. 
(This is by no means the highest rate of which the 
pigeon is capable.) Because this is an optimal con- 
dition, the rate prevails for some time. When the 
pigeon pauses for a few moments (at B), it cre- 
ates a condition which is not optimal for reinforce- 
ment. Responding is therefore not resumed for 
some time. Eventually another slow acceleration 
leads to the same high rate. When this is again 
broken (at C), another period of slow respond- 
ing intervenes, followed by another acceleration. 
Eventually the rate falls off in extinction. Al- 
though such a curve is complex, it is not disorderly. 
It is by no means random responding. Since no 
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external condition changes during the experimental 
period, the change in rate must be due to conditions 
altered by the bird’s own behavior. 

We can test the importance of the passage of 
time in accounting for behavior of this sort by giv- 
ing the pigeon an external “clock.” One such clock 
consists of a spot of light projected upon the key 
which the pigeon pecks. The spot marks time by 
changing size. At first it is only %th of an inch in 
length. It grows to %th of an inch at a given rate. 
The response to the key is reinforced when the spot 
is largest. When the pigeon returns to the key after 
reinforcement, the spot has again become small. 
Here is an external stimulus, then, roughly propor- 
tional to the time which has passed since the last 
reinforcement. Can it be used by the pigeon as a 
discriminative stimulus? 

To avoid a disturbing complication we must get 
the spot of light into the experiment before it func- 
tions as a clock. Suppose we begin by holding the 
spot still at its largest size, and build up the usual 
fixed-interval performance. In Fig. 4 the upper 

REINFORCED 



Fig. 4. First effect of allowing a spot of light to change 
size as a function of time. Upper curve: typical perform- 
ance under fixed-interval reinforcement with the spot of 
light at “large” as shown in the circles. Lower curve: first 
effect of changing the spot to “small” immediately after re- 
inforcement and allowing it to grow linearly during the fol- 
lowing interval of 10 min. Sample sizes of the spot are 
shown in the circles. 



Fic. 5. Progressive change in the pigeon’s response to an 
external clock. The series (1) through (S) illustrates the 
increasing effectiveness of the spot of light as a discrimina- 
tive stimulus. 

curve shows a standard sample. The spot was set 
at “large” and the record is typical of reinforce- 
ment at intervals of ten minutes. We now — for the 
first time — change the size of the spot, letting it 
begin at “small” to grow progressively larger dur- 
ing the interval. The spots in the circles above the 
lower record give sample readings of the clock at 
various positions. We observe that the pigeon is 
sensitively controlled by the size. When the spot is 
small it is most unlike its accustomed size, and the 
rate is almost zero. As the spot grows, the simi- 
larity increases and the rate rises. As the spot 
reaches its final standard size, the rate has reached 
or exceeded the value at which responses have been 
reinforced. Such a curve is not the effect of the 
passage of time; it is the effect of stimulus gen- 
eralization from large spots to smaller ones. 

Eventually, however, the correlation between the 
size of the spot and the passage of time is felt. The 
pigeon begins, so to speak, to “tell time.” In Fig. 5 
a series of records show the progress of a pigeon in 
learning to use the clock projected upon the key. 
Groups of three intervals each have been selected 
from a single experimental record. In Record 1 the 
curvature is already somewhat sharper than in the 
preceding figure. As the pigeon is repeatedly ex- 
posed to the changing spot and is reinforced only 
when the spot is large, these gradients become 
sharper still. By the time Record S is reached, the 
pigeon is not responding for approximately the first 
seven or eight minutes out of each ten. By that 
time the spot has reached a size very close to opti- 
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Fig. 6. Typical performances with the clock moving at 
different speeds. The clock completes the cycle from “small” 
to “large” in a given number of minutes as marked. 

mal and responding then begins and soon reaches 
a very high rate. 

Eventually the pigeon characteristically waits 
fully eight out of the ten minutes and responds at 
a rate of four or five responses per second during 
the remaining part of the interval. It has formed 
a very precise size discrimination. This would be 
the result without an added clock if the pigeon had 
what we call a precise “sense of time,” but it is ob- 
vious that the unamplified passage of time is very 
insignificant for the pigeon compared with a physi- 
cal clock of this sort. 

The extent of the control exercised by the size of 
the spot is beautifully illustrated if we withhold 
further reinforcement while allowing the clock to 
run, repeating cycle after cycle of the growth of the 
spot from small to large during extinction. The 
pigeon continues not to respond during all sizes of 
the spot except those close to the value which has 
previously obtained at reinforcement. As repeated 
responses go unreinforced, however, the amount of 
responding to the high value progressively de- 
creases. The extent of the control exercised by the 
spot can be shown in many other ways. We dis- 
covered one of these by accident. Our experiments 
are automatic, and our apparatus is used 24 hours 
of the day, When we reached the laboratory one 
morning, we found that a pigeon had not responded 
all night long. Investigation showed that through 
an oversight the clock had not been started. The 
spot had remained at its smallest size for IS hours. 
During this time the' pigeon had not made a single 
response to the key. 

At the other extreme, we can show the enormous 


stimulating power of the clock stopped at its opti- 
mal size. In a typical experiment the pigeon re- 
sponded nearly 2,000 times during a single 10- 
minute interval with the clock set at “large.” This 
extraordinary stimulus control far exceeds that ob- 
tained through the discrete presentation of dis- 
criminative stimuli. A continuously varying stimu- 
lus may possibly form the basis for a more sensi- 
tive psychophysical technique. 

When time has been, so to speak, externalized in 
this way, it may be manipulated. For example, our 
clock may be made to run fast or slow. In one ex- 
periment, various “speeds of time” were introduced 
at random in successive intervals. The clock might 
complete one cycle in, say, 6 minutes, at the end of 
which time a response would be reinforced; the 
next cycle might require 16 minutes; and so on. 
The extent of the control exercised over the bird’s 
behavior is seen in Fig. 6, where typical perform- 
ances for a range of clock speeds between one cycle 
in 3 minutes and one cycle in 32 minutes are shown. 
The rate of responding is roughly the same for a 
given size of spot regardless of speed of change. 
The curve at 32 minutes is obviously not approxi- 
mately 10 times as high as that at 3 minutes, how- 
ever, as it should be if the control by the spot were 
strictly equivalent in both cases. 

It is also possible to run externalized time back- 
ward. Our first experiment of this sort was also an 
accident. The bird was being studied with a 3- 
minute clock and was responding as shown at the 
left in Fig. 7. The next day, through an oversight, 
the clock was run backward. The spot began large 



Fig. 7. The effect of the clock running backwards. Left 
curve: standard performance with the clock operating as in 
earlier graphs. Right curve: record obtained when the 
clock was accidentally reversed. The control exercised by 
the size of the spot completely masks any effect of the pas- 
sage of time and inverts the standard curve. Successive re- 
inforcements with the clock at “small” soon destroy this 
control. 
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and grew small. The first three segments of the 
second curve in Fig. 7 are essentially inversions of 
the segments of the other curve. Since the bird 
was now reinforced when the spot was small, how- 
ever, a new pattern quickly arose. The curve be- 
comes essentially linear and at a later stage the 
usual performance with a clock developed. 

We may eliminate the effect of time by adopting 
a different schedule, in which reinforcement is still 
controlled by a clock, but the intervals are varied, 
roughly at random, within certain limits and with a 
given mean. In such a case the bird cannot predict, 
so to speak, when the next reinforcement is to be 
received. This is called variable-interval reinforce- 
ment. The effect is a uniform rate of responding 
with great stability, which may be maintained for 
many hours. During a single experimental period 
of fifteen hours a bird responded 30,000 times. To- 
ward the end of the record there was one pause ap- 
proximately one minute long but otherwise the bird 
did not pause longer than fifteen seconds at any 



Fig. 8. Fixed-ratio reinforcement. The ratio (r) is 200 
responses per reinforcement. 

time during the fifteen hours. During this period 
the bird received less than its daily ration of food. 

We turn now to an entirely different type of 
schedule. The moment at which a response is to 
be reinforced may be determined by the behavior 
of the organism itself. For example, we may re- 
inforce every fifth response, every fiftieth response, 
or every two-hundredth response. We call this 
“fixed-ratio reinforcement.” In industry, it is 
called piece-work pay. The pigeon’s behavior 
under such a schedule is not too difficult to inter- 
pret. Figure 8 shows a short segment of a charac- 
teristic performance. A response is reinforced every 
time the pigeon completes a group of 200 responses. 
Just as we represented a fixed-interval reinforce- 
ment by drawing vertical lines on our cumulative 
graph, so here we may represent fixed-ratio rein- 
forcement with a series of horizontal lines. When- 


ever the curve reaches one of these lines a response 
is reinforced, no matter how much time has elapsed. 

The results depend upon the size of the ratio. 
For ratios which may be easily maintained — for ex- 
ample, a ratio of 100:1 for the pigeon — the curves 
are essentially straight lines of high slope. Im- 
mediately after each reinforcement, however, a low 
rate of responding prevails which may be extended 
into long delays when the ratios are high. The 
transition from a low to a high rate between rein- 
forcements is sometimes of such a nature that the 
curve shows a smooth gradient as in Fig. 8. Other- 
wise the change from no responding to rapid re- 
sponding is usually abrupt. The high rate which 
prevails when the organism is responding appears to 
be due to another source of stimulation available 
under fixed-ratio reinforcement. In addition to a 
clock the pigeon presumably has a "counter” which 
tells it how many responses it has made since the 
previous reinforcement. 

An increase in its counter reading may be im- 
mediately reinforcing to the pigeon. One way to 
test this is to add an external counter comparable 
to the external clock. The spot of light on the key 
is made to grow, not with the passage of time, but 
with thp accumulation of responses. If the pigeon 
does not respond, the spot remains stationary. 
With each response it grows by a small amount. 
The effect of this externalized counter is dramatic. 
In one experiment the pigeon was being reinforced 
approximately every 70 responses. It was proceed- 
ing at an over-all speed of about 6,000 responses 
per hour. As soon as a spot of light was added to 
the key, in such a way that it grew from “small” 
to “large” as the effect of 70 responses, the rate 
went up almost immediately to 20,000 responses 
per hour. Pauses after reinforcement disappeared. 
Obviously the pigeon’s own “counter” is much less 
effective than the spot of light. It is possible to 
carry a pigeon to a high ratio without introducing 
appreciable pauses after reinforcement, but this 
process is slow and must be carried out with great 
care, presumably because the pigeon must be made 
sensitive to changes in its own counter. 

We can prove that the pigeon is, so to speak, 
counting its responses by setting up a two-valued 
schedule of reinforcement. We reinforce the fiftieth 
response after the preceding reinforcement or the 
two hundred and fiftieth, and we arrange our pro- 
gram in such a way that there is no indication in 
advance of which ratio is to prevail. In such a 
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Fig, 9. Two-valued reinforcement. Ratios of 50:1 and 
250:1 in random order. 

case, the pigeon, develops a steplike curve appro- 
priate to a ratio of 50:1. But it shows this, of 
course, even when the ratio is actually 250:1. In 
Fig. 9, for example, the segments at A, B, and C 
show either three or four waves which are the 
gradients prevailing under a reinforcement of 50:1. 
The pigeon begins as if the ratio were to be 50:1, 
but after 60 or 70 responses have been completed 
there is a marked decrease in rate which can be ex- 
plained only by assuming that the bird “knows the 
score.” A short period of slow responding follows. 
This gives way to a second gradient, again roughly 
of the order prevailing under 50:1 reinforcement. 
This may be followed by a third or even fourth 
gradient before reinforcement is received at the 
two-hundred-and-fiftieth response. If, as at D, we 
simply withhold all reinforcements, an extinction 
curve emerges in the form of a series of waves, 
averaging approximately 50 responses each. This 
cannot be due to the mere passage of time, since 


time does not show a wavelike character. It can- 
not be due to a discrimination based upon the rate 
of responding, because this should lead to long 
segments at a high rate as in both fixed-interval 
and fixed-ratio reinforcement. We have, then, to 
take into account a third source of automatic 
stimulation at the moment of reinforcement pro- 
vided by a “counter.” 

We can eliminate the “counter” by randomizing 
a schedule of many different ratios. In a typical 
experiment with what we may call “variable-ratio 
reinforcement,” a response was reinforced on the 
average every one hundred and ten responses, but 
in actual practice the very next response or a re- 
sponse as many as 500 responses later might be 
reinforced. The schedule produced a rate of re- 
sponding of approximately 12,000 responses per 
hour, which was sustained for long periods of time 
without any of the oscillations in rate characteristic 
of fixed-ratio reinforcement. This variable-ratio 
schedule is familiar to everyone, because it is the 
fundamental feature of all gambling devices. The 
pigeon making 12,000 responses per hour is not far 
removed from the pathological gambler. Variable- 
ratio reinforcement engages and holds the behav- 
ior of the organism with particular power. The 
magnitude of its control is seen when we extinguish 
the response. Figure 10 is an extinction curve ob- 
tained after the variable-ratio reinforcement just 
described. The curve has been broken into con- 
secutive segments in order to avoid undue reduc- 
tion. It begins with a long run of approximately 
7,500 responses during which there is no appreci- 
able retardation. The remainder of the curve is 
also illuminating. After short periods of slow re- 



Fig. 10. Extinction after variable-ratio reinforcement. The mean ratio was 110 responses pet 
reinforcement. The record has been broken into segments. 
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sponding the pigeon returns again and again to the 
original rate which, as the prevailing condition at 
previous reinforcements, tends to perpetuate itself. 

frequency of responding as an 
experimental datum 

Intermittent reinforcement is widely used in the 
control of human behavior. Many different kinds 
of wage systems illustrate it, and the schedules 
characteristic of gambling devices play a power- 
ful role. Almost all the complex behavior which 
we used to speak of as representing the higher men- 
tal processes arises from differential reinforcement 
which is necessarily intermittent, and we cannot 
evaluate such processes until the contributions of 
the schedules themselves have been discovered. 
This material is presented here, however, merely to 
exemplify the kind of result which follows when 
one takes probability — or, more immediately, fre- 
quency — of response as a subject matter. The fol- 
lowing points seem to be justified. 

1 . Frequency of response is an extremely orderly 
datum. The curves which represent its relations to 
many types of independent variables are encourag- 
ingly simple and smooth. 

2. The results are easily reproduced. It is sel- 
dom necessary to resort to groups of subjects at 
this stage. The method permits a direct view of 
behavioral processes which have hitherto been only 
inferred. We often have as little use for statistical 
control as in the simple observation of objects in 
the world about us, If the essential features of a 
given curve are not readily duplicated in a later ex- 
periment — in either the same or another organism — 
we take this, not as a cue to resort to averages, but 
as a warning that some relevant condition has still 
to be discovered and controlled. In other words, 
the uniformity of our results encourages us to turn, 
not to sampling procedures, but to more rigorous 
experimental control. 

3 . As a result of ( 2 ) the concepts and laws which 
emerge from this sort of study have an immediate 
reference to the behavior of the individual which is 
lacking in concepts or laws which are the products 
of statistical operations. When we extend an ex- 
perimental analysis to human affairs in general, it 
is a great advantage to have a conceptual system 
which refers to the single individual, preferably 
without comparison with a group. A more direct 
application to the prediction and control of the 


individual is thus achieved. The study of fre- 
quency of response appears to lead directly to such 
a system. 

4. Frequency of response provides a continuous 
account of many basic processes. We can follow 
a curve of extinction, for example, for many hours, 
and the condition of the response at every moment 
is apparent in our records. This is in marked con- 
trast to methods and techniques which merely sam- 
ple a learning process from time to time, where the 
continuity of the process must be inferred. The 
samples are often so widely spaced that the kinds 
of details we see in these records are missed. 

5. We must not forget the considerable advan- 
tage of a datum which lends itself to automatic ex- 
perimentation, Many processes in behavior cover 
long periods of time. The records we obtain from 
an individual organism may cover hundreds of 
hours and report millions of responses. We char- 
acteristically use experimental periods of eight, ten, 
or even fifteen hours. Personal observation of such 
material is unthinkable. 

6. Perhaps most important of all, frequency of 
response is a valuable datum just because it pro- 
vides a substantial basis for the concept of prob- 
ability of action — a concept toward which a sci- 
ence of behavior seems to have been groping for 
many decades. Here is a perfectly good physical 
referent for such a concept. It is true that the 
momentary condition of the organism as the tangent 
of a curve is still an abstraction — the very abstrac- 
tion which became important in the physical sci- 
ences with Newton and Leibnitz. But we are now 
able to deal with this in a rigorous fashion. The 
superfluous trappings to be found in traditional 
definitions of terms like habit, attitude, wish, and 
so on, may be avoided. 

The points illustrated here in a small branch of 
the field of learning apply equally well to other 
fields of behavior. Frequency of response has al- 
ready proved useful in studying the shaping of new 
responses and the interaction between responses of 
different topography. It permits us to answer such 
a question as: Does the emission of Response A 
alter the probability of Response B, which resem- 
bles A in certain ways? It has proved to be a use- 
ful datum in studying the effect of discriminative 
stimuli. If we establish a given probability of re- 
sponse under Stimulus A, we can discover the prob- 
ability that the response will be made under Stimu- 
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lus B, which resembles A. The question “Is red as 
different from orange as green is from blue?” is 
quite meaningful in such terms. Pattern discrimi- 
nation and the formation of concepts have been 
studied with the same method. 

Frequency of response is also a useful datum 
when two responses are being considered at the 
same time. We can investigate the behavior of 
making a choice and follow the development of a 
preference for one of two or more stimuli. The 
datum has proved to be especially useful in study- 
ing complex behavior in which two or more re- 
sponses are related to two or more stimuli — for ex- 
ample, in matching color from sample or in select- 
ing the opposite of a sample. Outside the field of 
learning considerable work has been done in the 
fields of motivation (where frequency of response 
varies with degree of deprivation), of emotion 
(where, for example, rate of responding serves as 
a useful baseline in observing what we may call 
“anxiety”), of the effects of drugs (evaluated, for 
example, against the stable baseline obtained under 
variable-interval reinforcement), and so on. One 
of the most promising achievements has been an 
analysis of punishment which confirms much of the 
Freudian material on repression and reveals many 
defects in the use of punishment as a technique of 
control. 

The extension of such results to the world at 
large frequently meets certain objections. In the 
laboratory we choose an arbitrary response and 
hold the environment as constant as possible. Can 
our results apply to behavior of much greater va- 
riety emitted under conditions which are constantly 
changing? If a certain experimental design is nec- 
essary to observe frequency, can we apply the re- 
sults to a situation where frequency cannot be de- 
termined? The answer here is the answer which 
must be given by any experimental science. Labo- 
ratory experimentation is designed to make a proc- 
ess as obvious as possible, to separate one process 
from another, and to obtain quantitative measures. 
These are the very heart of experimental science. 
The history of science show's that such results can 
be effectively extended to the v'orld at large. We 
determine the shape of the cooling curve only with 
the aid of the physical laboratory, but we have 
little doubt that the same process is going on as 
our breakfast coffee grows cold. We have no evi- 
dence for this, however, and probably could not 


prove it under genuine breakfast-table conditions. 
What we transfer from the laboratory to the casual 
world in which satisfactory quantification is impos- 
sible is the knowledge that certain basic processes 
exist, that they are lawful, and that they probably 
account for the unpleasantly chaotic facts with 
which we are faced. The gain in practical effec- 
tiveness which is derived from such transferred 
knowledge may be, as the physical sciences have 
shown, enormous. 

Another common objection is that if we identify 
probability of response with frequency of occur- 
rence, we cannot legitimately apply the notion to 
an event which is never repeated. A man may 
marry only once. He may engage in a business 
deal only once. He may commit suicide only once. 
Is behavior of this sort beyond the scope of such 
an analysis? The answer concerns the definition of 
the unit to be predicted. Complex activities are not 
always “responses” in the sense of repeated or re- 
peatable events. They are composed of smaller 
units, however, which are repeatable and capable 
of being studied in terms of frequency. The prob- 
lem is again not peculiar to the field of behavior. 
Was it possible to assign a given probability to the 
explosion of the first atomic bomb? The probabili- 
ties of many of the component events were soundly 
based upon data in the form of frequencies. But 
the explosion of the bomb as a whole was a unique 
event in the history of the world. Though the prob- 
ability of its occurrence could not be stated in terms 
of the frequency of a unit at that level, it could still 
be evaluated. The problem of predicting that a 
man will commit suicide is of the same nature. 

SUMMARY 

The basic datum in the analysis of behavior has 
the status of a probability. The actual observed 
dependent variable is frequency of response. In an 
experimental situation in which frequency may be 
studied, important processes in behavior are re- 
vealed in a continuous, orderly, and reproducible 
fashion. Concepts and laws derived from such data 
are immediately applicable to the behavior of the 
individual, and they should permit us to move on 
to the interpretation of behavior in the world at 
large with the greatest possible speed. 
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